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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )I3 Responsive to communication(s) filed on 29 September 2003 . 
2a)D This action is FINAL. 2b)E3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quay/e, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) |3 Claim(s) 1-118 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) H Claimfs) 20-23 .49-52.81 .83-85.87-95 and 101-103 is/are allowed. 

6) M Claim(s) 1-19.24.25.28-48.53.56-64.66.68-80.82.86.96-100.104 and 109-118 is/are rejected. 

7) M Claimfs) 26.54 and 105-108 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) IE The specification is objected to by the Examiner. 

10) 13 The drawing(s) filed on 24 November 2000 is/are: a)l3 accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

11) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) M Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
aM AH b)D Some * c)D None of: 

1 .13 Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 
Specification 

1. The abstract of the disclosure is objected to because it is longer than 25 lines or 150 
words. Correction is required. See MPEP § 608.01(b). 

2. The disclosure is objected to because of the following informalities: 

Each section is not clearly labeled, for example no distinction between the Background 
and Summary. 

Appropriate correction is required. 

3. The lengthy specification has not been checked to the extent necessary to determine the 
presence of all possible minor errors. Applicant's cooperation is requested in correcting any 
errors of which applicant may become aware in the specification. 

Claim Objections 

4. Claims 60 and 63 are objected to because of the following informalities: 

As to claims 60 and 63 are dependent on one independent claim while refer to limitation 
of another independent claim. Please delete references to claims 1 and 30 and physically include 
limitation that are intended to be in claims 60 and 63. 

Appropriate correction is required. 
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Claim Rejections - 35 USC § 101 



5. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or 
any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and 
requirements of this title. 

Claims 27, 55, 65 and 67 are rejected under 35 U.S.C. 101 because the claimed invention 
is directed to non- statutory subject matter. 

As to claim 27, the invention discloses non-functional descriptive material; please refer to 
the phrase on line 2 stating, "... outputting a signal carrying data defining, . . " A signal, per se, 
which is nothing more than an abstract idea, in order for the functionality of a data signal to 
be realized, it must be tangibly embodied on a "computer readable medium". Examiner 
suggests canceling claim 27. 

As to claims 65 and 67, the invention discloses non-functional descriptive material; 
please refer to the phrase on line 1 stating, "A signal carrying processor implementable 
instructions". A signal, per se, which is nothing more than an abstract idea, in order for the 
functionality of a data signal to be realized, it must be tangibly embodied on a "computer 
readable medium". Examiner suggests canceling claims 65 and 67. 



Claim Rejections - 35 USC § 112 



The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 



The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 
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6. Claim 24 is rejected under 35 U.S.C 1 12, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which applicant regards as 
the invention. 

Claim 24 recites the limitation "when the derived value for a voxel" in lines 12-13. There 
is insufficient antecedent basis for this limitation in the claim. 

Claim Rejections - 35 USC §102 

7. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

8. Claims 1-19, 25, 28-48, 53, 56-64, 66, 68-80, 82, 86, 96-100, 104 and 109-118 are 
rejected under 35 U.S.C. 102(b) as being anticipated by an article entitled "Rapid Octree 
Construction from Image Sequences" by Szeliski. 

As to claim 1 , Szeliski discloses a method of operating an image processing apparatus for 
processing image data representing images of an object taken from a plurality of different camera 
positions, the method comprising the steps of: 

(a) determining the viewing volume for each camera position (page 23, column 2, lines 
20-22 and 34-41, wherein for each viewpoint of the camera a conic volume is determined); 



• # 
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(b) determining the volume bounded by the intersection of the viewing volumes (page 
23, column 2, lines 20-23 and 34-41, wherein the conic volumes are intersected as each image 
arrives); and 

(c) setting the bounded volume as an initial space for use in deriving a representation of 
the three-dimensional surface of the object using said images (page 23, column 1, section 1, lines 
1-6 and page 24, Section 2, lines 1-3). 

As to claim 2, Szeliski discloses a method according to claim 1, further comprising the 

step: 

(d) of dividing the bounded volume into voxels to form an initial voxel space (page 24, 
Section 2, lines 1-3 and page 24, section 2, column 2, lines 1-4, wherein the initial voxel space 
consists of a single large cube). 

As to claim 3, Szeliski discloses a method according to claim 2, further comprising: 

(e) determining, for each voxel that is not occluded by another voxel, the area 
corresponding to that voxel in each image in which that voxel is visible (page 24, section 3, 
paragraph 2, lines 10-13); 

(f) comparing characteristics of each of the image areas corresponding to the same voxel 
(page 25, section 4); 

(g) removing a voxel in response to the characteristics of the image areas corresponding 
to that voxel being inconsistent (page 24, section 2, paragraph 1 and section 3, paragraph 1, 
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wherein the white cubes correspond to inconsistent voxel and they are remove after each 
revolution); and 

(h) repeating steps (e) to (g) until all non-occluded voxels having inconsistent 
characteristics have been removed (page 24, section 3, paragraph 1, wherein after each 
revolution each voxel is subdivided in to 8 voxel, wherein during the course of the revolution the 
cubes are designated as being white and are removed at the end of the revolution). 

As to claim 4, please refer to rejections made for claims 1-3 above. 

As to claim 5, Szeliski discloses a method according to claim 1 , wherein the step of 
determining the viewing volume for a camera position comprises determining the viewing 
volume using data representing a camera focal point and a camera imaging area for that camera 
position (page 28, section 7. 1). 

As to claim 6, Szeliski discloses a method according to claim 5, wherein the step of 
determining the viewing volume includes projecting straight lines from the focal point through 
points on the boundary of the imaging area (page 28, section 7. 1). 

As to claim 7, Szeliski discloses a method according to claim 1, wherein the step of 
defining the intersection of the viewing volumes comprises: 
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(i) determining an initial intersection of the viewing volumes of first and second camera 
positions (page 24, section 3, paragraph 1, wherein a sequence of viewing volumes are compared 
against the proceeding one to construct a 3D model of the object being viewed); 

(ii) determining the intersection of that intersection with another viewing volume and 
setting that intersection as the current intersection (page 24, section 3, paragraph 2, wherein with 
each image the 3D model is carved away); and 

(iii) repeating steps (i) and (ii) until the viewing volumes for all camera positions have 
been considered (page 24, section 3, paragraph 1, wherein during one revolution each camera 
position is considered). 

As to claim 8-15, please refer to rejections made for claims 1-3 and 5-7 above because no 
new limitations have been presented in claims 8-15 that have not already been addressed. 

As to claim 16, Szeliski discloses, in an image processing apparatus having a processor 
for processing image data representing images of an object taken from a plurality of different 
camera positions, a method of processing image data to derive a representation of a three- 
dimensional surface of the object, the method comprising the steps of: 

(a) defining an initial volume containing the object surface as an initial space formed of 
voxels (page 24, Section 2, lines 1-3 and page 24, section 2, column 2, lines 1-4, wherein the 
initial voxel space consists of a single large cube); 



Application/Control Number: 09/7 18,413 Page 8 

Art Unit: 2625 

(b) accessing data representing images of the object recorded at different camera 
positions with respect to the object (page 24, section 3, paragraph 1, wherein a sequence of 
silhouettes corresponds to images at different camera positions); 

(c) checking to see if a voxel meets at least one criterion by projecting that voxel into at 
least one of the images (page 24, section 3, paragraph 2 and page 25, section 4, wherein criterion 
corresponds to determining wherein voxel is inside or outside the silhouette); 

(d) if the voxel does not meet said at least one criterion, dividing the voxel into subsidiary 
voxels (page 24, section 2, paragraph 1, lines 1-3 and page 24 and 25, section 3, wherein if the 
voxel is gray it is dividing into more voxel); and 

(e) then checking to see if the subsidiary voxels meets at least one criterion by projecting 
the subsidiary voxels into at least one of the images (page 24 and 25, section 3, wherein once the 
voxels are divided the processes starts over in an effort to refine the voxels even more). 

As to claim 17, Szeliski discloses a method according to claim 16, further comprising 
deciding that a sub- voxel does not form part of the three-dimensional surface and so should be 
removed if the sub- voxel does not meets said at least one criterion (page 24, section 2, paragraph 
1, lines 1-3 and page 24 and 25, section 3, wherein if the voxel is not part of the 3D surface it is 
deemed a white voxel and removed) 

As to claim 18, Szeliski discloses a method according to claim 16, further comprising 
repeating steps c, d and e for any sub- voxel that does not meet said at least one criterion (page 24 
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and 25, section 3, wherein once the voxels are divided the processes starts over in an effort to 
refine the voxels even more). 

As to claim 19, Szeliski discloses a method according to claim 16, wherein the at least 
one criterion comprises any one or more of the following: 

1) the colour variance in a pixel patch to which the voxel projects in an image has a value 
lower than a predetermined value; 

2) the difference in colour or average colour between pixel patches to which the voxel 
projects in different images has a standard deviation less than a predetermined value; and 

3) the voxel is not partially occluded by a voxel or subsidiary voxels of smaller size than 
the voxel (page 24 and 25, section 3, wherein the voxel will only be designated white if it is not 
at least partially occluded by another voxel of any size, it would be considered gray if is 
occluded). 

As to claim 25, Szeliski discloses a method according to claim 4, wherein the step of 
determining the area corresponding to a voxel in an image comprises projecting the voxel into 
each image (page 24, section 3, paragraph 2, lines 10-13). 

As to claim 28, Szeliski discloses a method according to claim 4, which further comprises 
providing a computer storage medium storing data defining the initial space or the voxel 
representation of the 3D object surface (page 24, column 1, lines 4-5). 
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As to claim 29, Szeliski discloses a method according to claim 4, further comprising 
generating texture data for rendering onto the representation of the 3D object surfaces (page 3 1 , 
section 10, paragraph 3). 

As to claim 30-36, please refer to rejections made for claims 1-7, respectively, above- 
As to claim 37-44, please refer to rejections made for claims 1-3 and 5-7 above because 
no new limitations have been presented in claims 37-44 that have not already been addressed. 

As to claims 45-48, please refer to rejections made for claims 16-19, respectively, above. 
As to claim 53, please refer to rejections made for claim 25 above. 
As to claim 56, please refer to rejections made for claim 28 above. 
As to claim 57, please refer to rejections made for claim 29 above. 

As to claims 58, 59, 61 and 62, Szeliski discloses a method according to claim 16, which 
further comprises: 

Accessing data representing a further image of the object recorded at different camera 
position; and then repeating steps of claim 16 using that further image (page 24, section 3, 
paragraph 2, lines 10-13). 

As to claim 60 and 63, please refer to rejections made for claim 1 above. 

As to claim 64 and 66, Szeliski discloses a storage medium carrying processor 
implementable instructions for causing processing means to carry out a method in accordance 
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with claim 1 or 30 (page 23, Abstract, lines 8-15, wherein computation inherently means that the 
invention includes a storage medium carrying processor implementable instructions). 

As to claim 68-80, 82, 86, 96-100 and 109-1 18, please refer to rejections made for claims 
1-3, 5-7, 16-19, 25 and 28 above because no new limitation have been presented in the claims 
that have not already been addressed. 

As to claim 104, please refer to rejections made for claim 25 above. 

Allowable Subject Matter 

9. Claims 20-23, 49-52, 81, 83-85, 87-95 and 101-103 are allowed. 

10. Claims 26, 54 and 105-108 are objected to as being dependent upon a rejected base 
claim, but would be allowable if rewritten in independent form including all of the limitations of 
the base claim and any intervening claims. 

1 1 . Claim 24 would be allowable if rewritten or amended to overcome the rejection(s) under 
35 U.S.C. 1 12, second paragraph, set forth in this Office action. 

12. The following is an examiner's statement of reasons for allowance: As to claims 20, 49, 
81, 83-85, 87-89 91, 93, 95, 101 and 103 none of the prior art teach or fairly suggests the 
limitations of deriving from the compared characteristics a value representing the degree of any 
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inconsistency between the characteristics of the image areas corresponding to the given voxel 
and sub-dividing a voxel into subsidiary voxels in response to the derived value exceeding a 
threshold value and repeating steps (c) to (e) for each subsidiary voxel, in combination with the 
other limitations of the claims. As to claims 23, 51 , 90, 92, 94 and 102 none of the prior art 
teach or fairly suggests the limitation of determining a colour space value for each pixel of each 
area where each colour space value encompasses a range of pixel colour values, comparing the 
colour space values for each of the image areas corresponding to the same voxel, and removing 
the voxel only if the image areas do not share at least one colour space value. However, US 
Patent 6,363,170 to Seitz et al. ("Seitz") discloses deriving a color value for voxels represented 
in each image, determining a degree of inconsistency between corresponding color values of 
images and based on a threshold deciding whether or not to add the voxel to the final 3D 
representation, Seitz does not teach or fairly suggest that based on the result of comparing the 
value with the threshold that the voxel is then divided into subsidiary voxel or completely 
removed for the already in place 3D representation of the object. 

Any comments considered necessary by applicant must be submitted no later than the 
payment of the issue fee and, to avoid processing delays, should preferably accompany the issue 
fee. Such submissions should be clearly labeled "Comments on Statement of Reasons for 
Allowance " 

Conclusion 

13. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 
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US Patent 5,917,937 to Szeliski et al. discloses determining a volume boundary through 
viewpoint intersection. 

US Patent 6,373,977 to Culbertson discloses determining a volume boundary through 
viewpoint intersection and discusses how to handle color. 

US Patent 6,351,572 to Dufour discloses determining a volume boundary through 
viewpoint intersection and octree reconstruction. 

US Patent 6,407,738 to Wakabayashi discloses determining a volume boundary through 
viewpoint intersection and octree reconstruction. 

US Patent 6,563,499 to Waupotitsch et al discloses determining a volume boundary 
through viewpoint intersection and octree reconstruction. 

US Patent 6,574,360 to Berdardini et al. discloses determining a volume boundary 
through viewpoint intersection and octree reconstruction. 

US Patent 5,189,626 to Colburn discloses determining a volume boundary through 
viewpoint intersection and octree reconstruction. 

US Patent 6,088,035 to Sudarsky et al. discloses determining a volume boundary through 
viewpoint intersection and octree reconstruction. 

"How Far 3D Shapes Can Be Understood from 2D Silhouettes" by Laurentini, Aldo 
discloses determining a volume boundary through viewpoint intersection. 

"3D Shape Reconstruction using Volume Intersection Techniques" by Carr et al. 
discloses determining a volume boundary through viewpoint intersection. 
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"Inferring the Shape of the Real Object from the Object Reconstructed by Volume 
Intersection" by Laurentini, Aldo discloses determining a volume boundary through viewpoint 
intersection. 

"The Visual Hull Concept for Silhouette-Based Image Understanding" by Laurentini, 
Aldo discloses determining a volume boundary through viewpoint intersection. 

14. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Aaron W Carter whose telephone number is (703) 306-4060. The 
examiner can normally be reached on 7am - 3:30 am (Mon. - Fri.). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bhavesh Mehta can be reached on (703) 308-5246. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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